Papers with Word alignment
A Bidirectional Transformer Based Alignment Model for Unsupervised Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning word alignment include statistical word aligners (e.g. GIZA++) Existing word alignment models employ a target-to-source attention mechanism which can provide rough word alignments but with a low accuracy. |
| Approach: | They propose a bidirectional Transformer based alignment model for unsupervised learning of the word alignment task. |
| Outcome: | The proposed model outperforms both previous neural word alignment approaches and the popular statistical word aligner GIZA++ on three word alignment tasks. |
When do Contrastive Word Alignments Improve Many-to-many Neural Machine Translation? (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to improve pre-training for many-to-many neural machine translation use manual cleaning of bilingual dictionaries, which are unavailable for most language pairs. |
| Approach: | They propose a word-level contrastive objective to leverage word alignments for many-to-many neural machine translation (NMT) Empirical results show that this leads to 0.8 BLEU gains for several language pairs. |
| Outcome: | Empirical results show that the proposed objective leads to 0.8 BLEU gains for several language pairs. |
End-to-End Neural Word Alignment Outperforms GIZA++ (2020.acl-main)
Copied to clipboard
| Challenge: | Word alignment was once a core unsupervised learning task in natural language processing . but word alignment still plays an important role in interactive applications of neural machine translation, such as annotation transfer and lexicon injection. |
| Approach: | They propose to use a Transformer model to train an unsupervised word alignment model. |
| Outcome: | The proposed method outperforms GIZA++ on three data sets and is tightly integrated and does not affect translation quality. |
Word Alignment by Fine-tuning Embeddings on Parallel Corpora (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing work on word alignment has focused on unsupervised learning on parallel text. |
| Approach: | They propose to combine pre-trained contextualized word embeddings with multilingually trained language models to achieve competitive results on word alignment tasks. |
| Outcome: | The proposed model outperforms state-of-the-art models on five language pairs and can train multilingual word aligners that can obtain robust performance on different language pairs. |
Third-Party Aligner for Neural Word Alignments (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work shows that word alignment can be competitive . |
| Approach: | They propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. |
| Outcome: | The proposed approach can find more accurate word alignments and delete wrong alignments, leading to better performance than the current best third-party word aligner. |
Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing word alignment models capture few interactions between input sentence pairs, which severely degrades the word alignment quality. |
| Approach: | They propose to model deep interactions between input and target sentences using a two-stage training framework to train the model. |
| Outcome: | The proposed model achieves the state-of-the-art (SOTA) performance on four out of five language pairs. |
Mask-Align: Self-Supervised Neural Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Word alignment is an important task in many natural language processing tasks. |
| Approach: | They propose a self-supervised word alignment model that takes advantage of the full context on the target side. |
| Outcome: | The proposed model outperforms previous unsupervised models and obtains state-of-the-art results on four language pairs. |
PMI-Align: Word Alignment With Point-Wise Mutual Information Without Requiring Parallel Training Data (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that using contextualized embeddings from pre-trained multilingual language models could give us high quality word alignments without the need of parallel training data. |
| Approach: | They propose a method which uses contextualized embeddings from pre-trained language models to extract word alignments without parallel training. |
| Outcome: | The proposed method outperforms rival methods on five out of six language pairs. |
Improving Word Alignment Using Semi-Supervised Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing word alignment methods rely on labeled data, but augmenting training with pseudo-labeled data improves performance. |
| Approach: | They propose a semi-supervised framework to improve word alignment methods . they use pseudo-labeled data from multilingual encoder models as word aligners . |
| Outcome: | The proposed framework outperforms the current state-of-the-art binary alignment method on word alignment datasets. |